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Abstract We study the convergence properties of alternating proximal minimization algorithms for (nonconvex) 
functions of the following type: 

L{x, y) = f(x) + Q(x, y) + g(y) 

where / : R n — > Ru{+oo} and g : R m — > RU{+oo} are proper lower semicontinuous functions and Q : R n x R m — > 
R is a smooth C (finite valued) function which couples the variables x and y. 
The algorithm is defined by: 

(x ,yo) £ R n x R m given, (x k ,y k ) -> (x k+1 ,y k ) -> (x k+1 ,y k+1 ) 

x k+1 £ argmin{L(M,y fc ) + axrju - x k \\ 2 : u £ R"} 
y k+1 £ argmm{L(x k+1 ,v) + -L-|| w - y k f : v <= R m } 

Note that the above algorithm can be viewed as an alternating proximal minimization algorithm. Alternating 
projection algorithms on closed sets are particular cases of the above problem: just specialize / and g to be the 
indicator functions of the two sets and take Q(x,y) = \\x — y\\ 2 . 

The novelty of our approach is twofold: first, we work in a nonconvex setting, just assuming that the function 
L is a function that satisfies the Kurdyka-Lojasiewicz inequality. An entire section illustrates the relevancy of 
such an assumption by giving examples ranging from semialgebraic geometry to "metrically regular" problems. 
Secondly, we rely on a new class of alternating minimization algorithms with costs to move which has recently 
been introduced by Attouch, Redont and Soubeyran. 

Our main result can be stated as follows: Assume that L has the Kurdyka-Lojasiewicz property and that the 
sequence (x k ,y k ) k £N is bounded. Then the trajectory has a finite length and, as a consequence, converges to a 
critical point of L. This result is completed by the study of the convergence rate of the algorithm, which depends 
on the geometrical properties of the function L around its critical points (namely the Lojasiewicz exponent). 
As a striking application, we obtain the convergence of our alternating projection algorithm (a variant of the 
von Neumann algorithm) for a wide class of sets including in particular semialgebraic and tame sets, transverse 
smooth manifolds or sets with "regular" intersection. 

Key words Alternating minimization algorithms, alternating projections, Lojasiewicz inequality, convergence 
rate, o-minimal structures, tame optimization, proximal algorithm. 



1 Introduction 

In this paper, we will be concerned by the convergence analysis of alternating minimization algorithms 
for (nonconvex) functions of the following type: 

L(x,y) = f(x)+Q(x,y)+g(y) (1) 

where / : R" — > R U {+00} and g : R m KU {+00} are proper lower semicontinuous functions and 
Q : 1" x R m — > R is a smooth C 1 (finite valued) function which couples the variables x and y. Alternating 
projection algorithms on closed sets are particular cases of the above problem: just specialize / and g to 
be the indicator functions of the two sets and take Q(x, y) = \\x — y\\ 2 . 
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The novelty of our approach is twofold: first, we work in a nonconvex setting, just assuming that 
the function L is a function that satisfies the Kurdyka-Lojasiewicz inequality, see [32], [53], [20]. As it 
has been established recently in [101 [HI Q~2] , this assumption is satisfied by a wide class of nonsmooth 
functions called functions definable in an o- minimal structure (see Section [4]). Semialgebraic functions 
and (globally) subanalytic functions are for instance definable in their respective classes. 

Secondly, we rely on a new class of alternating minimization algorithms with costs to move which has 
recently been introduced by Attouch, Redont and Soubeyran [5] and which has proved to be a flexible 
tool, see [3]. Our study concerns the convergence of the following algorithm (discrete dynamical system) 

{xq,Do) erxl" 1 given, {x k ,y k ) -> (x k+ i,y k ) -> (x k+1 ,y k+ i) 



Note that the above algorithm can be viewed as an alternating proximal minimization algorithm. 

Our main result (theorem [T2]) can be stated as follows: Assume that L has the Kurdyka-Lojasiewicz 
property and that the sequence (x k , y k ) k ^ is bounded. Then the trajectory has a finite length and, as a 
consequence, converges to a critical point of L. This result is completed by the study of the convergence 
rate of the algorithm, which depends on the geometrical properties of the function L around its critical 
points (namely the Lojasiewicz exponent). 

There are two rich stories behind this problem. One concerns the alternating minimization algorithms, 
the other the use of the Lojasiewicz inequality in nonsmooth variational analysis. Our study is precisely 
at the intersection of these two active fields of research. Let us briefly delineate them and so put to the 
fore the originality or our approach. 

1. Let us recall the classical result (1980) due to Acker and Prestel [2J: let TL be a real Hilbert space 
and / : H i— ► R U {+oo}, j:W^IRU {+°°} two closed convex proper functions. Consider the sequence 
(xk,yk)keN generated by the classical alternating minimization algorithm 



J x k+ i = argmin{/(u) + | || u - y k ||^: u S H}, 
| y k+ i = argmin{#(v) + § || x k+1 - v ||^: v G H}. 

Then, the sequence (x k ,y k ) ke fq weakly converges to a solution of the joint minimization problem onHxH 



if we assume that the minimum point set is nonempty. 

Acker and Prestel's theorem provides a natural extension of von Neumann's alternating projection 
theorem [2H] for two closed convex nonempty sets of the Hilbert space H (take / = Sc ± , g = <5c 2 the 
indicator functions of the two sets). Note that the two sets may have an empty intersection, in which case 
the algorithm provides sequences converging to points in the respective sets which are as close to each 
other as possible. This fact has proved of fundamental importance in the applications of the method to 
the numerical analysis of ill-posed inverse problems. A rich litterature has been devoted to this subject, 
one may consult [7], [15], [33] • 

Indeed, in recent papers [5], H] a new class of alternating minimization algorithms with costs to move 
has been introduced. The most original feature of these algorithms is the introduction of the cost-to- move 
terms ||u — Xfc|j 2 , \\v — y k \\ 2 - In the modelling of real world dynamical decision processes (here two agents 
play alternatively) these terms reflect an anchoring effect: the decision x k +\ is anchored to the preceding 
decision x k , the term \\u — x k \\ 2 is the cost to change when passing from decision x k to u. Similarly, 
\\v — Uk\\ 2 is the cost to change when passing from decision y k to v. The importance of these terms is 
crucial when decisions, performances are seen as routines, ways of doing. They take account of inertial 
effects and frictions [6j. 

From the algorithmic point of view, the introduction of these cost-to-move terms presents many 
advantages: They confer to the dynamic some stabilization and robustness properties. Moreover they 



x k+ i G argmin{L(w, y k ) + ^\\u- x k \\ 2 : u g R n }, 
y k+ i e argmm{L(a; fc+ i,u) + J-\\v - y k \\ 2 : v e M m }. 
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naturally appear when discretizing steepest descent dynamical systems for the related joint function (here 
the function L). As a result, the above algorithms are dissipative and enjoy nice convergent properties. 
Note that the cost-to-move terms asymptotically vanish and do not affect the type of equilibrium which 
is finally reached (indeed they allow to select one of them in the case of multiple equilibria) . 

2. Proximal algorithms can be viewed as implicit discretizations of the continuous steepest descent 
method. In the convex case, most results concerning the continuous dynamical system (steepest de- 
scent) can be converted to the discrete one (proximal algorithm) and vice versa. A deep and general 
understanding of these connections requires the introduction and the study of the differential inclusion 

±{t) e -df(x(t)), t > o 

where df denotes the subdifferential of the lower semicontinuous function / : H i— > R U {+00}. 

Besides the modelling aspects, our special interest in the proximal method for functions involving 
analytic features comes from the recent development of the study of the steepest descent method for 
such functions. In his pioneering work on real-analytic functions [22] . [23], Lojasiewicz provided the basic 
ingredient, the so-called "Lojasiewicz inequality" , that allows to derive the convergence of all the bounded 
trajectories of the steepest descent to critical points. Given a real-analytic function / : R™ — > R and a 
critical point a G R ra the Lojasiewicz inequality asserts that there exists some 9 £ [i, 1J such that the 
function |/ — f(a)\ e || V/|| _1 remains bounded around a. Similar results have been developed for discrete 
gradient methods (see PQ) and nonsmooth subanalytic functions (see [TOl ITT] )- 

In the last decades powerful advances relying on an axiomatized approach of real-semialgebraic/real- 
analytic geometry have allowed to set up a general theory in which the basic objects enjoy the same 
qualitative properties as semialgebraic sets and functions [TTJ [3U] • In such a framework the central 
concept is the one of o-minimal structures over R. Basic results are recalled and illustrated in Section 
14.2.51 Following van den Dries [17], functions and sets belonging to such structures are called definable 
or tameE). Extensions of Lojasiewicz inequality to definable functions and applications to their gradient 
vector fields have been obtained by Kurdyka [20] , while nonsmooth versions have been developed in 
|12| . The corresponding generalized inequality is hereby called the Kurdyka-Lojasiewicz inequality (see 
Definition O Section @J). 

Keeping in mind the close connection between the continuous steepest descent and the proximal 
algorithms it is then natural to study the convergence of such algorithms for functions with "tame" 
features. A first step has recently been accomplished by Attouch and Bolte [3] who proved the convergence 
of the proximal algorithm for nonsmooth functions that satisfy the Lojasiewicz inequality around their 
generalized critical points. 

Let us now come to our situation which involves two decision variables x and y and a bivariate function 

L(x,y) = f(x)+Q(x,y)+g(y). (2) 

We are interested in studying splitting (alternate) algorithms whose trajectories converge to critical points 
of L. 

The first key point of our approach is to assume that L satisfies the Kurdyka-Lojasiewicz inequality. 
The relevancy of this assumption is illustrated by many examples including uniformly convex functions, 
convex functions enjoying growth conditions, semialgebraic functions, definable functions. Specific ex- 
amples related to feasibility problems are also described, they involve (possibly tangent) real-analytic 
manifolds, transverse manifolds (see (2Tj), semialgebraic sets or more generally tame sets... 

This being assumed, it is then natural to apply to L an algorithm which is as close as possible to 
the proximal algorithm in the product space. The proof of our main result consists in proving that the 
alternating proximal minimization algorithm is close enough to the product space-proximal algorithm 
(one has to control the correcting terms introduced by the alternating effect), which makes it share the 
nice convergence properties of this algorithm. 

5 The word tame actually corresponds to a slight generalization of definable objects. 
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As a striking application, we obtain the convergence of our alternating projection algorithm, which 
can be seen as a variant of the von Neumann algorithm (section 5.3). Being given two closed subsets 
C, D of R™ the latter is modelled on 

^ p ( x k + Vk \ 

y/c+i e P D I g J ' 

where Pc,Pd '■ R™ =4 R n are the projection mappings of C and D. The convergence of the sequences 
(xk), (yk) is obtained for a wide class of sets ranging from semialgebraic or definable sets to transverse 
manifolds or more generally to sets with a regular intersection. A part of this result is inspired by the 
recent work of Lewis and Malick on transverse manifolds [21] . in which similar results were derived. 

The paper is organized as follows: After the introduction, section 2 is devoted to recalling some 
elementary facts of nonsmooth analysis. This allows us to obtain in section 3 some first elementary 
properties of the alternating proximal minimization algorithm. Then, in section 4 we introduce the 
Kurdyka-Lojasiewicz inequality for lower semicontinuous functions and describe various classes of func- 
tions satisfying this property. The main convergence result is proved in section 5 together with an 
estimation of the convergence rate. The last section is devoted to some illustrations and applications of 
our results. 

2 Elementary facts of nonsmooth analysis 

The Euclidean scalar product of R n and its corresponding norm are respectively denoted by (•, •) and | • |. 
General references for nonsmooth analysis are [331 EH [25] . 

If F : R™ =$ R m is a point-to-set mapping its graph is defined by 

Graph F := {(x, y) G R" x R m : y G F(x)}. 

Similarly the graph of a real-extcnded-valued function / : R™ -»RU {+00} is defined by 

Graph / := {(x,s) G R" x R : s = f(x)}. 

Let us recall a few definitions concerning subdifferential calculus. 

Definition 1 ([28]) Let / : R" — > R U {+00} be a proper lower semicontinuous function. 

(i) The domain of / is defined and denoted by dom/ := {x G R n : f(x) < +00}. 

(ii) For each x E dom/, the Frechet subdifferential of / at x, written df(x), is the set of vectors 
x* G R™ which satisfy 

liminf - [f(y) - f{x) - (x*,y-x)] > 0. 

y x \\ x y\\ 

y^x 

If x $ dom/, then df(x) = 0. 

(iii) The limiting-subdifferential (|24j). or simply the subdifferential for short, of / at x G dom/, 
written df, is defined as follows 

d f{x) := {x* G R" : 3x n -» x, f(x n ) -» f(x), < G df(x n ) -» x*}. 

Remark 1 (a) The above definition implies that df(x) C df(x) for each x G R™, where the first set is 
convex and closed while the second one is closed. 

(b)(Closedness of df) Let (xk, x%) € Graph df be a sequence that converges to (x, x*). If f{xk) converges 
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to f(x) then (x,x*) G Graphd/. 

(c) A necessary (but not sufficient) condition for x S R™ to be a minimizer of / is 

df(x) 3 0. (3) 

A point that satisfies ([3]) is called limiting- critical or simply critical. The set of critical points of / is 
denoted by crit /. 

If K is a subset of R n and x is any point in R™ , we set 

dist (x, K) = inf{||iE — z|| : z € X}. 

Recall that if if is empty we have dist (x, K) = +oo for all x G R ra . Note also that for any real-extended- 
valued function / on R" and any x G R™, dist (0, df(x)) = mf{||x*|| : x* G df(x)}. 

Lemma 2 For any sequence Xk — > x such that f(xk) — > /(x), we have 

liminf dist (0,df (x k )) > dist (0,9/ (x)). (4) 

k — > + oo 

Proof. One can assume that the left hand side of (j4} is finite. Let x* k G df(xk) be a sequence such 
that liminf fc^+oo ||x£|| = liminffc^+oo dist (0, df(xk)). There exists a subsequence (x* k t) of (x£) such that 
linik'_>-|-oo ll^fe'll = liminf/ £ _ + + 00 Since {x%,) is bounded, one may assume that it converges to some 

x* . Using Remark [1] (b) , we see that x* G df(x). Thus df(x) is nonempty and lim||x£/|| = ||x*|| > 
dist (0,9/ (a;)). □ 

Partial subdifFerentiation Let L : M." x M. m ->KU {+00} be a lower semicontinuous function. When 
y is a fixed point in R m the subdifferential of the function L(-, y) at u is denoted by d x L(u, y). Similarly 
one can define partial subdifferentiation with respect to the variable y. The corresponding operator is 
denoted by d y L(x, •) (where x is a fixed point of M. n ). 
The following elementary result will be useful 

Proposition 3 Let L : R n x W n — * R U {+00} be a function of the form L(x, y) = f(x) + Q(x, y) + g(y) 
where Q : R 71 x R m — > R is a finite-valued Frechet differ entiable function and f : R™ R U {+00}. 
g : R m -4KU {+00} are lower semicontinuous functions. Then for all (x,y) G domi = dom/ x domg 
we have 

df(x) + V x Q(x,y) 
9g(y) + V y Q{x,y) 



9L(x,y) 



Proof. Observe first that we have dL(x,y) — d(f(x) + g(y)) + VQ(x,y), since Q is differentiable ( |28|, 
10.10 Exercice, p. 431]). Further, the subdifferential calculus for separable functions yields ([211 10.5 
Proposition, p. 426]) 9(/(x) + g(y)) — df(x) x dg(y). □ 

Normal cones, indicator functions and projections 

If C is a closed subset of R" we denote by 5c its indicator function, i.e. for all x G R ra we set 

Sc(x) = { ° if X G C ' 
y ' 1 +00 otherwise. 

The projection on C, written Pq, is the following point-to-set mapping: 
Pc 



c — ► Pc{x) '■= argmin — z\\ : z G C}. 

When C is nonempty, the closedness of C and the compactness of the closed unit ball imply that Pc(%) 
is nonempty for all x in R™. 
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Definition 4 (Normal cone) Let C be a nonempty closed subset of R™. 

(i) For any x G C the Frechet normal cone to C at x is denned by 

N c (x) = {ve E" : (w.y - s) < o(z - y), Vy G C}. 
When i^Cwe set JVc(af) = 0- 

(ii) The (limiting) normal cone to C at x is denoted Nc(x) and is defined by 

u G A^c(ic) ^ 3x/c — > x, v k G Nc{x k ),Vk — > u. 

Remark 2 (a) For igC the cone Nc(x) is closed but not necessarily convex. 

(b) An elementary but important fact about normal cone and subdifferential is the following 

5<5 C = 7V C . 

(c) Recall that for a point-to-set mapping F : M." =t E m , its generalized inverse i* 1-1 : R m =4 R™ is 
defined by x G F^ 1 (y) y G F{x). Writing down the optimality condition associated to the definition 
of Pcj w e obtain 

P c C (J + JVc) -1 , 

where J is the identity mapping of R" . 



3 Alternating proximal minimization algorithm 

Let L : R" x R™ 1 — > R U {+00} be a proper lower semicontinuous function. Being given (xo,yo) G 
R n x R m we consider the following type of alternating discrete dynamical system: (x^, y k ) — > (%k+i > ?/fe) ~~ * 
(xfe+i,yfe+i) 



{x k+ i G argmin {L(u,y k ) + — \\u - x k \\ 2 : u G R"} (5) 
G argmin {^(xfc+i, w) + ^-||w-2/fc|| 2 : v G R m }, (6) 

where (Afe)fegN) (Mfc)feeN are positive sequences. 
We make the following standing assumptions 

J {Hi) infRn xR , n L > — 00, 

\ (^2) the function L(-,yo) is a proper function. 



Lemma 5 Assuming (Hi), {H2), the sequences (x k ), (y k ) are correctly defined. Moreover: 

(i) The following estimate holds 

L(x k+u y k+ i) < L(x k ,y k ) - — \\x k +i - x k \\ 2 - — ||y k+1 - y fe || 2 . (7) 

i\k ^k 

(ii) First-order optimality conditions read 

Xk+i - x k + X k d x L(x k+ i,y k ) 3 0, (8) 
Vk+i - Vk + [i k d y L(x k+ i, y k+ i) 3 0. (9) 

Proof. Since inf L > —00, H2 implies that for any r > 0, (u,v) G R™ x R m the functions u — > 
L(u,v) + ^ || u — u|| 2 and v — > L(u 7 v) + v\\ 2 are coercive. An elementary induction ensures then 

that the sequences are well defined and that (J7J) holds for all integer k. Hence (i) holds. 
The first-order optimality condition [28, Theorem 10.1] and the subdifferentiation formula for a sum of 
functions [28l Exercise 10.10] yield © and ©. □ 
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Remark 3 Without additional assumptions, like for instance the convexity of L, the sequences Xk, y k 
are not a priori uniquely defined. 

We consider the following additional assumptions: 

- Being given < r_ < r + , we assume that the sequences of stepsizes Xk and \ik belong to (r_,r+) for 
all k > 0. 

- (7^3) The restriction of L to its domain is a continuous function. 

Proposition 6 Assume that ((Hi), (W2), (H3) hold. Let (xk,y k ) be a sequence which complies with 
Denote by lo(xq, yo) the set (possibly empty) of its limit points. Then 
(i) L(xk,yk) is decreasing, 
(H) E " Xk\\ 2 + hk+i ~ VkW 2 < +00, 

(Hi) assuming moreover that L is of the form L(x, y) — f(x) + Q(x, y) + g(y), then lo(xq, y ) C crit L. 
If in addition (xk,yk) is bounded then 

(iv) uj(xo,yo) is a nonempty compact connected set, and 

d((xk,yk),u(xo,yo)) — > as k — > +00, 

(v) L is finite and constant on oj(xo,yo). 

Proof. Assertion (i) follows immediately from (O, while (ii) is obtained by summation of (JT]) for k = 
1,...,N: 

L(xn+i,VN+i) + r- ^2 \\ x k+i - Xk\\ 2 + \\Vk+i - ykW 2 ^ < L(x\,y\) < +00. (10) 

Let us establish (iii). Item (ii) implies that Xk+i — Xk and yk+i ~ yk tend to zero. Using jS]) and ([9]), 
we see that there exists {x* k ,y* k ) G {d x L(x k ,y k -i),dyL(x k ,y k )) = (df(x k ) + V x Q(x k , yk-i), dg(y k ) + 
V ' y Q{xk,yk)) that converges to zero as k goes to 00. If (x,y) is a limit point of {xk,yk), Lemma [S] and 
the closure of the graphs of df and dg show that dL(x, y) 3 0. 

Item (iv) follows by using (ii) together with some classical properties of sequences in W 1 . Assertion 
(v) is a consequence of (i) and (H3). □ 

4 Kurdyka-Lojasiewicz inequality 
4.1 Definition 

Let / : R" —* R U {+00} be a proper lower semicontinuous function that is continuous on its domain. 
For < ?/i < i]2 < +00, let us set 

[m < f < m] = {x eR n -.m < f(x) < m) ■ 

We recall the following convenient definition from [3]. 

Definition 7 (Kurdyka-Lojasiewicz property) The function / is said to have the Kurdyka-Lojasie- 
wicz property at a; € dom df if: 

There exist 77 G (Q,+oo], a neighborhood [f] U of x and a continuous concave function <f> : [0,77) — > M+ 
such that: 

• 0(0) = 0, 

• 4> is C 1 on (0, 77), 

• for all s g (0,77), (j)'(s) > 0. 



6 See Remark (gj (c) 
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and 



<P'(f(x) - /(x))dist (0, df(x)) > 1, whenever x belongs to U n [f(x) < f < f(x) + 77]. (11) 

Remark 4 (a) S. Lojasiewicz proved in 1963 22J that real-analytic functions satisfy an inequality of the 
above type with (ft(s) — s 1 ^ 9 where 9 £ [5,1). A nice proof of this result can be found in the monograph 
[15] . In a recent paper, Kurdyka has extended this result to functions definable in an o- minimal structure 
(see the next section). 

(b) Let / : R n — ► RU {+00} be a proper lower semicontinuous function. For any noncritical point x € R n 
of /, Lemma [2] yields the existence of e, 77 > and c such that 

dist (0, df(z)) > c> 

whenever x £ B(x, e) n [—77 < / < rj\. In other words / has the Kurdyka-Lojasiewicz property at x with 
4>{s) = c~ 1 s. 

(c) When a function has the Kurdyka-Lojasiewicz property, it is important to have nice estimations of 
r), U, 4>. We shall see for instance that some convex functions satisfy the above property with U = W 1 and 
77 = +00. The determination of tight bounds for the nonconvex case is a lot more involved. 

4.2 Examples of functions having the Kurdyka-Lojasiewicz property 

4.2.1 Growth condition for convex functions 

Let U C R™ and 77 £ (0, +00]. Consider a convex function / satisfying the following growth condition: 

3c > 0,r > l,Vx € U (1 [min/ < / < 77], f(x) > f(x) + cd(x, argmin f) r , 
where x € argmin / 7^ 0. Then / complies with (| 1 1 1) (for cf>(s) = r c~r s^) on U n [inf / < / < 77] (see 

cdd- 

4.2.2 Uniform convexity 

If / is uniformly convex i.e., satisfies 

f(y)>f(x) + (x*,y-x)+K\\y-xr, 

for all x, y £ M™, x* £ df(x) then / satisfies the Kurdyka-Lojasiewicz inequality on dom/ for <j>(s) — 

_ 1 1 
p K p sp . 

Proof. Since / is coercive and strictly convex, argmin / = {x} 7^ 0. Take y £ K™. By applying the 
uniform convexity property at the minimum point x, we obtain that 

f(y) >wmf + K\\y-x\\*. 

The conclusion follows from Section f4. 2. 11 

4.2.3 Morse functions 

A C 2 function / : W l —> R is a Morse function if for each critical point x of /, the hessian V 2 /(i) of / 
at x is a nondegenerate endomorphism of R" . 

Let i be a critical point of /. Using the Taylor formula for / and V/, we obtain the existence of a 
neighborhood U of x and positive constants ci , C2 for which 

\f(x) - f(x)\ < Cl \\x - x\\ 2 , \\Vf(x)\\ > calls - x\\, 

whenever x £ U. 

It is then straightforward to see that / complies to (fTTj) with a function cf> of the form <f>(s) = Cy^s, 
where c is a positive constant. 
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4.2.4 Metrically regular nonsmooth equations 

The example to follow is inspired by a result of [27) that was used to analyze the complexity of second- 
order gradient methods. 

Let us consider a mapping F : 1" — > R m . The function F is called metrically- regular on a subset C 
if there exists k > such that 

dist (x, < fcdist (y, F(x)) 

for all (x, y) G C x F(C). The coefficient fc is a measure of stability under perturbation of equations of 
the type -F(x) = f/. When F is C 1 a famous result, the so-called Lusternik-Graves theorem (see |19j). 
asserts that F is fc-metrically regular on C if and only if DF(x) is surjective for all x in C with 

||[Z>F(i)*] _1 || < fc, for all x in C, 

where DF(x) (resp DF*(x)) denotes the Frechet derivative of F at x (resp. the adjoint of DF(x)). 

Fundamental extensions to nonsmooth inclusions have been provided by many authors, see [191 128) 
and references therein. For the sake of simplicity we assume here that F is C 1 and fc-metrically regular 
on some subset C of R" . 

Let us consider the following nonlinear problem: find i£C such that F(x) = 0. 

Solving this problem amounts to solving 

min{/(a;) := -||F(x)f : x £ C}, 
P 

where p G [1, +oo). 

The function / is differentiable at each x G R n \/ _1 (0) with gradient V/(x) = ||F(x)||P- 2 L>F*(a;)F(a;). 
By using the metric regularity of F and Lusternik-Graves theorem we have 

||V/(x)|| > || [X>J^*] — 1 (sc) || — 1 ||F(aO|| p - 1 > k^p^fix)^ for all x eC\f 1 (0). 
In other words / has the Kurdyka-Lojasiewicz property at any x G int C (the interior of C) with 

U = C, T) = +oo, and <j)(s) = p? k s? , s G R. 

It is important to observe here that argmin / can be a continuum (in that case / is not a Morse 
function). This can easily be seen by taking m < n, b £ R™ 1 , a full rank matrix A £ jg mx " ano ; 
F(x) = Ax-b for all x in R n . 

4.2.5 Tame functions 

Semialgebraic functions : Recall that a subset of R" is called semialgebraic if it can be written as a 
finite union of sets of the form 

{x £ R" : pi(x) = 0, qt(x) < 0, i = 1, . . . 

where pj, qi are real polynomial functions. 

A function / : R™ — > R U {+00} is semialgebraic if its graph is a semialgebraic subset of R™ +1 . Such 
a function satisfies the Kurdyka-Lojasiewicz property (see QUI [12]) with (f>(s) = s 1 " 6 , £ [0, 1) HQ. This 
nonsmooth result generalizes the famous Lojasiewicz inequality for real-analytic functions [22j . 

Here are a few examples of semialgebraic functions: 

(a) R" 3 x -> ||Ax -6||^ + \\Cx - d\\ m , where A £ W nxn ,b£ R m , C £ W n ' xn ,d£ R m ', 

(b) R™ 3 x — > /(x) = sup ygC g(x,y) where g and C are semialgebraic. 

(c) R" x R™ 9 (x, y) — > L(x, j/) = |||x — y|| 2 + ^c(-t) + <5o(y) where C and D are semialgebraic subsets 
ofR". 
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Functions definable in an o-minimal structure over R. Introduced in [T7j these structures can be 
seen as an axiomatization of the qualitative properties of semialgebraic sets. 

Definition 8 Let O = {0„}„£N be such that each O n is a collection of subsets o/R n . The family O is 
an o-minimal structure over R 7 if it satisfies the following axioms: 

(i) Each O n is a boolean algebra. Namely G O n and for each A, B in O n , A U B , An B and R" \ A 
belong to O n . 

(ii) For all A in O n , AxM and R x A belong to O n +i- 

(Hi) For all A in O n +i, n(A) := {{x\, . . . ,x n ) G R" : (x\, . . . ,x n ,x n +\) G A} belongs to O n . 

(iv) For all i 7^ j in {1, . . . , n}, {(xi, . . . , x n ) G R n : Xi — Xj} G O n . 

(v) The set {(xi,X2) G R 2 : x\ < X2} belongs to 02- 

(vi) The elements of 0\ are exactly finite unions of intervals. 

Let O be an o-minimal structure. A set A is said to be definable (in O), HA belongs to O. A point 
to set mapping F : R™ =$ R m (resp. a real-extended-valued function / : R" — > R U {+00}) is said to be 
definable if its graph is a definable subset of R™ x R m (resp. R™ x R). 

Due to their dramatic impact on several domain in mathematics, these structures are intensively 
studied. One of the interest of such structures in optimization is due to the following nonsmooth extension 
of Kurdyka-Lojasiewicz inequality. 

Theorem 9 (|12j) Any proper lower semicontinuous function / : R n — > R U {+00} which is definable 
in an o-minimal structure O has the Kurdyka-Lojasiewicz property at each point of dome*/. Moreover 
the function <f> appearing in is definable in O. 

The concavity of the function <p is not stated explicitly in [T2]. The proof of that fact is however 
elementary: it relies on the following fundamental result known as the monotonicity Lemma (see |17j). 

(Monotonicity Lemma) Let k be an integer and f : L C R — > R a function definable in some o-minimal 
structure. There exists a finite partition of I into intervals I\, . . . ,I p such that the restriction of f to 
each Ii is C k and either strictly monotone or constant. 

Proof of Theorem [9] [concavity of <f>] Let i be a critical point of / and <j> a L function of / at x (see 
(jlip ). As recalled above such a function <f) exists and is definable in O moreover. When applied to <fi 
the monotonicity lemma yields the existence of r > such that <fi is C 2 with either <f" < or <fi" > 
on (0, r) (apply the monotonicity lemma to <fi"). If <f" > then 0' is increasing. Take so G (0, s). Since 
< <fi'(so) for all s G (0, sq), the function (f> in inequality (fTTj) can be replaced by i/j(s) = (f>'(so)s for 
allsG(0,s o ). □. 
The following examples of o-minimal structures illustrate the considerable wealth of such a concept. 

Example 1 (semilinear sets) A subset of R™ is called semilinear if it is a finite union of sets of the 
form 

{x G R™ : (Oj,x) = oti, (bi,x) < fa, i= 1, ... ,p}, 

where a,i, b% G R™ and on, Pi G R. One can easily establish that such a structure is an o-minimal structure. 
Besides the function <j) is of the form <j>{s) = cs with c > 0. 

Example 2 (real semialgebraic sets) By Tarski quantifier elimination theorem the class of semialge- 
braic sets is an o-minimal structure [51 [3] ■ 
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Example 3 (Globally subanalytic sets) (Gabrielov [18] ) There exists an o-minimal structure, denoted 
by R an , that contains all sets of the form {(x,t) E [-1, lfxK: f(x) = t} where / : [-1, 1]" -»K(n£ N) 
is an analytic function that can be extended analytically on a neighborhood of the square [—1,1]". The 
sets belonging to this structure are called globally subanalytic sets. As for the semialgebraic class, the 
function appearing in Kurdyka-Lojasiewicz inequality is of the form <p(s) = s 1 ^ , 9 E [0, 1) n Q. 

Let us give some concrete examples. Consider a finite collection of real-analytic functions fa : R™ — > K 
where i = 1, . . . ,p. 

- The restriction of the function / + = max^ fa (resp. /_ = mim/i) to each [—a,a] n (a > 0) is globally 
subanalytic. 

- Take a finite collection of real-analytic functions gj and set 

C={xeM n :f i (x) = 0,g j (x)<0}. 

If C is a bounded subset then it is globally subanalytic. Let now G : R' m x R" — > M be a real analytic 
function. The restriction of the following function to each [—a, a]" (a > 0) is globally subanalytic: 

f(x) = maxG(x,y). 



Example 4 (log-exp structure) (Wilkie, van der Dries) [301 Hi] There exists an o-minimal structure 
containing R all and the graph of exp : M — > R. This structure is denoted by R an ,exp- This huge structure 
contains all the aforementioned structures. One of the surprising specificity of such a structure is the 
existence of "infinitely flat" functions like x — > exp(— ^y). 

Many optimization problems are set in such a structure. When it is possible, it is however important 
to determine the minimal structure in which a problem is definable. 

This can for instance have an impact on the convergence analysis and in particular on the knowledge 
of convergence rates (see Section HT21 Theorem W5\ for an illustration). 

4.3 Feasibility problems 

Being given two nonempty closed subsets C, D of R", one introduces the following type of function: 

Lc,D(x,y) :=L(x,y) = -\\x - y\\ 2 + S c (x) + 5 D {y), 

where (x, y) € R™ x R n . 

Note that L is a proper lower semicontinuous function that satisfies 7ii, H.3 and H.2 for any y^ 6 D. 
Writing down the optimality condition we obtain that 

dL(x,y) = {(x-y + u,y-x + v) : u S N c (x),v E N D (y)}. 

This implies that 

dist (0, dL(x, y)) = (dist 2 {y - x, N c (x)) + dist 2 (x - y, N D (y)) ) 1 . (12) 



4.3.1 (Strongly) regular intersection 

The following result is a reformulation of pT , Theorem 17] where this fruitful concept was considered in 
relation with alternate projection methods. For the sake of completeness we give a proof avoiding the 
use of metric regularity and Mordukhovich criterion. 
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Proposition 10 Let C,D be two closed subsets o/R" and x G C fl D. Assume that 

~N c {x)nN D {x) = {0}. 
Then there exists a neighborhood U C R n x R™ of (i, x) and a positive constant c for which 

dist (0, dL c , D (x, y)) >c\\x- y\\ > 0, (13) 

whenever (x, y) G U H [0 < Lq,d < +oo]. 

In other words Lc,d has the Kurdyka-Lojasiewicz property at (x,x) with <p(s) — \\fs- 

Proof. We argue by contradiction. Let (xk, yk) € [0 < L < +oo] be a sequence converging to (x, x) such 
that 

dist(0,dLc,p(xk,yk)) < 1 

\\x k - Vk\\ ~ k + 1' 

In view of (fT2|) . there exists (uk,Vk) £ Nc(xk) x Nrj(yk) such that 

2 

Ikfe - J/fcirMllyfe - ^fc - "fell + Ikfe - 2/fe - «fe||] < TTT- 

K+1 

Let d S S" 1 " 1 be a cluster point of \\x k - yk\\^ 1 (yk - x k ). Since ||x fc - II —1 [(z/fc - ^fe) - u k] converges 
to zero, d is also a limit-point of ||a;fe — yk\\~ 1 Uk S Nc(xk)- Due to the closedness property of iVc, we 
therefore obtain d £ Nc{x). Arguing similarly with Vk we obtain that d £ —Nd(x). □ 

Remark 5 Observe that (fT3)) reads 

(dist 2 (y - x, N c (x)) + dist 2 (a: - y, JV D (y)) ) 4 > c\\x - y\\. 

4.3.2 Transverse Manifolds 

As pointed out in [21] a nice example of regular intersection is given by the smooth notion of transversality. 
Let M be a smooth submanifold of W 1 . For each x in M, T X M denotes the tangent space to M at x. 
The normal cone to M at x is given by 

Nm(x) — T X M ± . 

Two submanifolds of 1" are called transverse at x £ M n N if they satisfy 

T X M + T X N = R n . 

Due to the previous remark we have Nm{x) n Nn{x) = Nm(x) n [— Nn(x)] = {0}. And therefore Lm,n 
satisfies the Kurdyka-Lojasiewicz inequality near x with 4>{s) = c -1 -^, c > 0. 

The constant c can be estimated by means of a notion of angle [211 Theorem 18]. 

4.3.3 Tame feasibility 

Assume that C, D are two globally analytic subsets of R™. The graph of Lc,d is given by 

GraphL C!jD = Gn(Cxflx R), 

where G is the graph of the polynomial function (x, y) — > \\x — y\\ 2 . From the stability property of 
o-minimal structures (see Definition^ (i)-(ii)), we deduce that Lc,d is globally subanalytic. 

Let (x,y) £ C x D. Using the fact that the bifunction satisfies the Lojasiewicz inequality with 
</>(s) = s 1 ^ 6 (where 9 £ (0, 1]) we obtain 

(dist 2 (y-x,N c (x))+ dist 2 {x-y,N D {y))Y > \\x - y\\ 28 (14) 
for all (x, y) with x ^ y in a neighborhood of (x, y). 
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Remark 6 (a) One of the most interesting features of this inequality is that it is satisfied for possibly 
tangent sets. 

(b) Let us also observe that the above inequality is satisfied if C, D are real-analytic submanifolds of R" . 
This is due to the fact that B((x, y), r) n [C x D] is globally subanalytic for all r > 0. 

If C,D and the square norm function || • || 2 are definable in an arbitrary o- minimal structure, we of 
course have a similar result. Namely, 

fi{\\x - y\\ 2 ) (dist 2 (y - x, N c (x)) + dist 2 {x - y, N D (y)) ) * > 1, (15) 

for all (x, y) in a neighborhood of a critical point of L. 

Although (fl5| is simply a specialization of a result of [12] , we feel that this form of Kurdyka-Lojasiewicz 
inequality can be very useful in many contexts. 

Remark 7 (a) Note that if C, D are definable in the log-exp structure then (fT"5|) holds. This is due to 
the fact that the square norm is also definable in this structure. 

(b) If C, D are semilinear sets, Lc,d is not semilinear but it is however semialgebraic. 

5 Convergence results: alternating minimization and alternat- 
ing projection algorithms 

5.1 Lipschitz coupling 

This section is devoted to the convergence analysis of the proximal minimization algorithm introduced 
in Section G3 

Let us assume that L is of the form 

L(x,y) = f(x) + Q(x,y)+g(y) (16) 

where / : M" — > K U {+oo}, g : W n — > K U {+00} are proper lower semicontinuous functions and 
Q : W l x W n — » M is a C 1 (finite valued) function with VQ being Lipschitz continuous on bounded 
subsets of M" x R m . 

Lemma 11 (pre-convergence result) Assume that L satisfies (Hi), (H2) o,nd has the Kurdyka-Loja- 
siewicz property at z = (x,y). Denote by U,r) and 4> : [0, 77) :— > K the arguments appearing in Hll\) . Let 
e > be such that B(z,e) C U and set p = max(^-,2C 2 + -3-), where C is a Lipschitz constant o/VQ 

on B(z, e). 

Let (iCfc,2/fc) be a sequence such that (xo,2/o) G B(z,e) PI [L(z) < L < 77] and inffe g ^ L(xk, yu) > ^(^)- 

//• 

2\/pr + 4> (L(x ,yo) - i(^) ) + V r+{L(x Q ,yo) - L(z)) + ^\\x - x\\ 2 + \\y - y\\ 2 < e, 

then 

(i) The sequence {(xk,yk)} lies in B(z,e). 

(ii) The following estimate holds 

+00 

J2 y/\\xi+i-Xi\\ 2 + \\y i+1 -yi\\ 2 < 2{J-pr+cp{l k ) + ^T k ) + j\\x k - x \\ 2 + \\y k - y|| 2 , 

i—k 

where l k = L(x kl y k ) - L(z). 

(ii) The sequence (x kl y k ) converges to a critical point of L. 
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Proof. With no loss of generality one may assume that L(z) = (replace if necessary L by L — L(z)). If 
L(xk, Uk) = L(z) for some fc > 0, the fact that inf^gN L(xk, Vk) > L(z) together with inequality imply 
that Xi = Xk, Hi = Vk for all i > fc. Since the indices k for which ||xfc+i — Xk\\ + \\yk+i — Uk\\ =0 have no 
impact on the asymptotic analysis we can assume that (xk, yk) G [0 < L < 77] and ||£Cfc+i — Xk\\ + ||yfe+i — 
Uk\\ > for all k > 0. Assume that k > 1 and let x\ G df(xk), l/t G dg(yk) be such that 

.ta; = Xfe-i - Afe(a;^ + VxQ(xk,Vk-i)), 
Vk = yk-i - Hk(yi + VyQ(x k ,yk))- 



Let us prove by induction that (xk,yk) G B(z,e) for all fc > 0. The result is obvious for k = 0. Assume 
that fc > 1 and that (xi,yi) G e) for all i G {1, . . . , fc}. 

Let i G {1, . . • , fc}, and set = L(xk,yk)- Since 0' > 0, implies that 

c(>'(k)(li - h+i) > - Xif + \\y i+1 - y t \\ 2 }. 

2r+ 



By using the concavity of the function 



m - Hh+i) > ^-[\\xi+i - x l \\ 2 + \\y i+ i - yi \\ 2 }. (17) 
2r+ 

For each i set Vi — (x* + V ' x Q{xi,yi),y* + V y Q(xi,yi)) G dL(xi,yi). Using the definition of x* and 
y* we obtain successively 

ll^ll 2 = II 7^-1 - Xi) + V x Q(xi, yi ) - V,Q(a; 4 ,y l _i)|| 2 + \\ — ( Vi - y;_i)|| 2 

\ Mi 

< ^r{2\\ Xi ~ x.-iH 2 + \\ yi - y t - 1 f) + 2C 2 \\y t - y^f 

< max(^-,2C 2 + ^-) [\\ Xi - x^f + \\ yi - yi-xf] (18) 

Set p = max(-|-, 2C 2 + -7-). The induction assumption implies that (f>'(k) > dist (0, dL(xi, yi))^ 1 ■ Hence 
inequalities (fTS"]) and (fTT|) yield 

2 ^r + [0ft) - > !:rT!r + ?rT'Jp - (19) 

Let us observe a simple fact: if ^- < r for some real numbers a, 6 > 0,r > 0, then a < \(b + r). This 
follows indeed from the fact that 2ab < a 2 + b 2 . When applying this inequality to (fTi?)) , we obtain 



y/pr+[<KU) - Hh+i)} + 5 VON - ^-ill 2 + lb« - z/i-i II 2 ] > v¥i+i - ^ll 2 + \\yi+i - y*\\ 2 - (20) 

After summation over all i = 1, . . . , fc, we therefore obtain 



Vpr+^i) + - a=o|| 2 + - yo\\ 2 > 2^pr + [4>(h) - 4>(h+i)] + y/\\ Xl - x Q \\ 2 + \\ Vl - y \\ : 

k 

> ^2 vPw ~ ^n 2 + ibi+i ~ ^n 2 - 

i=0 

By (O, we have ||xi — xo|| 2 + ||yi — ?/o|| 2 < r+[lo — ii] < r+ lo- We finally obtain 

k 

2^-pr+^h) + > Vll^+i-^P + ll^+i-y.ll 2 - 
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Put Zi = (xi,yi). Using successively the triangle inequality and the above inequality 



\z k+1 ~Z\\ < \\z k +l - Zq\\ + \\Zq - Z\\ 
k 

i=0 



< 2^r +( j)(h) + y/r+h + \\zo-z\\ 

< e. 

The conclusion follows readily. □ 
This Lemma has two important consequences 

Theorem 12 (convergence of bounded sequences) Assume that L satisfies (Hi), (H2), (H3) and 
has the Kurdyka-Lojasiewicz property at each point. 
If the sequence (x k ,y k ) is bounded then 



^2 \\x k+1 - x k \\ + \\y k+1 - y k \\ < +00, 



fe=i 

and as a consequence (x k ,y k ) converges to a critical point of L. 

Proof. Let z be a limit-point of (x k ,y k ) for which we denote by e, 77, <f> the associated objects as defined 
in pT|) . Note that Proposition [5] implies that z is critical. By Lemma [5] and (H3), L(x kl y k ) converges 
to L(z). Since max((/)(l k ) — L(z), \\(x k ,y k ) — z\\) admits as a cluster point, we obtain the existence of 
k > such that 

y/pr + (f>(l ko - L(z)) + 2y/r + l kn + ||(a; feo ,y feo ) - z\\ < e. 
The conclusion is then a consequence of Lemma 1111 □ 

Theorem 13 (local convergence to global minima) Assume that L satisfies (Hi), (H2) and has 
the Kurdyka-Lojasiewicz property at z. 

If z is a global minimum of L, then there exist e, r\ such that 

\\xq — z\\ < e, mini < L(xo,yo) < minL + 77 

implies that (x k ,y k ) has the finite length property and converges to (x*,y*) with L(x*,y*) = L(z). 

Proof. A straightforward application of Lemma QT] yields the convergence result. To see that L(x k ,y k ) 
converges to minL, it suffices to observe that L(x*,y*) is a critical value in [mini, minL + 771). □ 

Remark 8 Theorem gives new insights into convex alternating methods: first it shows that the finite 
length property is satisfied by many convex functions (e.g. convex definable functions), but it also relaxes 
the usual quadraticity assumption on Q that is required by usual methods, see [1] and references therein. 

5.2 Convergence rate 

Theorem 14 (rate of convergence) Assume that (x k ,y k ) converges to (a; 00 ,?/ 00 ) and that L has the 
Kurdyka-Lojasiewicz property at (x 00 ^ 00 ) with (f>(s) = cs 1 ^ , 9 G [0,1), c > 0. Then the following 
estimations hold 

(i) If 9 = 0, the sequence (x k ,y k ) ke jq converges in a finite number of steps. 

(ii) If 9 £ (0, i] then there exist c > and t £ [0, 1) such that 

\\(x k ,y k ) - (x°° ,y™)\\ <cr k . 
(Hi) If 9 £ (5,1) then there exists c > such that 

\\(x k ,y k )-(x^,y^)\\<ck-^. 
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Proof. The notations are those of the previous proof and for simplicity we assume that Ik — > 0. Assume 
first 9 — and take fc > 0. Either l k = inf{/i : i > fc} hence (x k ,y k ) is stationary (cf. proof of Lemma 
[TTj) . or dist (0, dL(xk, yk)) > c where c is a positive constant. In that case inequality (O reads 

h+i — h < — c 

where c' > 0. We therefore obtain that Zfc D = inf^ U for some integer feg. 

Assume that > 0. For any fe > 0, set Afe = YliLk vH^i+i — Xi\\ 2 + \\yi+i — Vi\\ 2 which is finite by 
Theorem [T2l Since Afe > ^|| a;fe x°°\\ 2 + \\yk — y°°\\ 2 , it is sufficient to estimate Afe. With no loss of 
generality we may assume that Afe > for all fc > 0. From (|20[) we obtain after summation 

^J5r+ <j)(l k ) + -(A fe _i - A fe ) > Afe. 
Using the Kurdyka-Lojasiewicz inequality and (|18p we obtain 

cp{l k )=cl k - d < c[(c(l-0))- 1 dist(O,9L(x fc , 2 / fc ))] i ^ 



< Ky/\\xk ~ x k -i\\ 2 + \\y k - y k -i\\ 2 

< K(A fc _ x - Afe) 1 ^, 

where K is a positive constant. Combining the above results we obtain 

if(A fc _i - A fc )^ + ^(A fe _i - Afe) > A fe where K > 0. 

Sequences Afe satisfying such inequalities have been studied in [21 Theorem 2]. Items (ii) and (iii) 
follow from these results. □ 

5.3 An application to alternating projection 

In this section we consider the special but important case of bifunctions of the type 

L c , D (x,y) = 5 c (x) + ^\\x - y\\ 2 + 5 D {y) 7 (x,y)eR n , 

where C,D are two nonempty closed subsets of K™. In this specific setting, the proximal minimization 
algorithm [©, ©] reads 

x k+ i G argmin {-\\u - y k \\ 2 + rr-||« - x k \\ 2 : u G C} 
I l\ k 

y k+1 G argmin {-\\v - x k+1 \\ 2 + - — \\v - y k \\ 2 : v G D}, 

z zn k 

thus we obtain the following alternating projection algorithm 

'X k X x k + y k 



Xfc+i G P ( 



c 



Afe 1 + 1 



M fc y k + x k+1 

yk+l G F D 33— 

V H + 1 

The following result illustrates the interest of the above algorithm for feasibility problems. 

Theorem 15 (convergence of bounded sequences) Assume that the bifunction Lq.d has the Kur- 
dyka-Lojasiewicz property at each point. Then any bounded sequence (x k , y k ) converges to a critical point 
ofL. 

(local convergence) Assume that the bifunction Lq.d has the Kurdyka-Lojasiewicz property at (x* ,y*) 
and that \\x* — y*\\ = min{||a; — y\\ : x G C, y G D}. 

If (xo,yo) is sufficiently close to (x*,y*) then the whole sequence converges to a point (x°°,y°°) such 
that \\x°° - y°°\\ = min{||x- y\\ : x G C,y G D}. 
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Proof. The first point is due to Theorem [T21, while the second follows from a specialization of Theorem IT51 

to l c ,d- n 



This variant of the von Neuman alternating projection method yields convergent sequences for a wide 
class of sets (see Section [4.3.3)) . e.g. when C, D, || • || 2 are definable in the same o-minimal structure or 
when the intersection C n D is regular. 

Note also that convergence rates results of Section 15.21 can be applied to the following fundamental 
cases 

- C,D scmilinear, semialgebraic, globally subanalytic or real-analytic submanifolds of W l . 

- C n D regular {e.g. transverse manifolds). 

For this last case the convergence is of linear type, i.e. \\xk — a?oo|| < Cirf, \\yk — Uoo\\ < C2 r| where 
q > and n G (0, 1). 
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